A Markov Decision Process Approach for Decentralized UAV Formation Path Planning

Fleet coordination and formation flight for Unmanned Aerial Vehicles (UAVs) are challenging and important problems that have received significant attention in recent years. In this paper, we propose a decentralized approach based on a Markov Decision Process (MDP) to ensure the control and formation flight of UAVs. We present a methodology for planning trajectories online that enables UAVs to maintain formation geometry while avoiding no-fly zones and reaching a desired goal area. By leveraging the dynamic model of fixed-wing UAV within the MDP formalization, we can provide optimal reference values for the UAV low-level controller. Furthermore, by harnessing the capabilities of MDPs to handle uncertainty, we can consider the behavior of nearby UAVs taking advantage of the predicted state vectors shared among them. Therefore, by exploiting the UAV dynamic model, and the estimation of the possible trajectories of the other UAVs, we can ensure collision-free and feasible actions for the UAV in a decentralized way. This approach has been validated and tested through simulations involving various scenarios.