Reinforcement Learning Aided Secure UAV Communications against Roaming Adversaries

The rapid integration of Unmanned Aerial Vehicle(UAV) with existing network infrastructure brings enormous benefits to users, however it also introduces vulnerabilities to information security. In this paper we investigate a UAV-aided secure communication system, where a UAV is deployed to transmit confidential information to a ground user. Specifically, a mobile eavesdropper is moving in the vicinity of the ground user, attempting to intercept legitimate data transmissions. Its unpredictable trajectory poses as an extra security risk to UAV data transmission. Driven by this security challenge, we aim to optimize the UAV trajectory to maximize user average secrecy rate. Due to the fast changing environment caused by unpredictable movement of the eavesdropper, this nonconvex optimization problem is difficult to solve. Instead, we propose an online algorithm leveraging the Q-learning framework to deliver online decisions on UAV trajectory. With the help of carefully designed reward signals, the agent is able to learn an effective policy with desirable learning outcomes. Numerical results validate the effectiveness of the proposed algorithm, and shed light on learning outcomes with a variety of learning parameters.