Defending against jamming and interference for Internet of UAVs using cooperative multi-agent reinforcement learning with mutual information

The Internet of Unmanned Aerial Vehicles (I-UAVs) is expected to execute latency-sensitive tasks, but limited by co-channel interference and malicious jamming. In the face of unknown prior environmental knowledge, defending against jamming and interference through spectrum allocation becomes challenging, especially when each UAV pair makes decisions independently. In this paper, we propose a cooperative multi-agent reinforcement learning (MARL)-based anti-jamming framework for I-UAVs, enabling UAV pairs to learn their own policies cooperatively. Specifically, we first model the problem as a model-free multi-agent Markov decision process (MAMDP) to maximize the long-term expected system throughput. Then, for improving the exploration of the optimal policy, we resort to optimizing a MARL objective function with a mutual-information (MI) regularizer between states and actions, which can dynamically assign the probability for actions frequently used by the optimal policy. Next, through sharing their current channel selections and local learning experience (their soft Q-values), the UAV pairs can learn their own policies cooperatively relying on only preceding observed information and predicting others' actions. Our simulation results show that for both sweep jamming and Markov jamming patterns, the proposed scheme outperforms the benchmarkers in terms of throughput, convergence and stability for different numbers of jammers, channels and UAV pairs.