Maneuvering Target Tracking of UAV in Urban Environment with PPO-MT

Tracking a maneuvering target using UAVs in an uncertain environment poses significant challenges. In order to solve the problem of maneuvering target tracking of UAV in urban environment, this paper proposes the proximal policy optimization with memory and transformer(PPO-MT) algorithm based on PPO algorithm. The PPO-MT algorithm uses a memory unit and transformer to make full use of the information from multiple time steps of the UAV and consider the importance weight of that information. Compared to the PPO algorithm, the PPO-MT algorithm demonstrated a remarkable increase of 41.7% in convergence speed and a notable improvement of 16.5% in success rate. Moreover, the average reward and maximum reward obtained by the PPO-MT algorithm during the training process are higher than the PPO algorithm.