Reinforcement Learning-Based Prescribed-Time Optimal Formation Control for Multiagent Systems

This paper addresses the problem of prescribed-time optimal formation control for nonlinear multi-agent systems (MAS) via reinforcement learning (RL). First, a new finite horizon performance cost function is constructed, which incorporates specified time and terminal constraints, thereby it is useful for embedding the prescribed convergence time performance into optimal control framework. Based on the cost function, RL-based optimal formation controller is subsequently devised under actor-critic structure, by incorporating terminal constraint errors into the critic update law, it ensures that the optimal value function can be approximated while satisfying terminal constraints. Then, the prescribed-time optimal formation controller is designed by integrating an adjustment function with the devised optimal formation controller, with which, the formation tracking errors are ensured to converge to the origin within specified time. The proposed approach provides some advantages such as system model information is not required, optimal performance is achieved with satisfied terminal constraints, and asymptotic rather than bounded prescribed-time formation control is achieved. Finally, the effectiveness of the proposed strategy is validated through simulation examples. Note to Practitioners—Prescribed-time formation control of multi-agent systems is the theoretical foundation of numerous industrial processes, thereby motivating the research work in this paper. The existing formation control methods predominantly focus on achieving formation objective with satisfied stead-state performance, but usually neglect the equivalent important transient performance and optimal performance, that restricts the role of formation control method in engineering applications. This paper proposes a reinforcement learning based approach that enables multiagent systems to simultaneously achieve formation within user-defined time constraints and optimal performance. Moreover, the actor-critic reinforcement learning framework eliminates system’s reliance on precise dynamical model, demonstrating enhanced adaptability for industrial applications. It should be noted that current research has not considered the effective utilization of communication resources. Future work will incorporate event-triggered control approach to improve communication efficiency while maintaining formation control objective.