Consensus Q‐Learning for Multi‐agent Cooperative Planning

This chapter proposes consensus‐based multi‐agent Q‐learning (MAQL) to address the bottleneck of the optimal equilibrium selection among multiple types. It briefly introduces the adaption mechanism of single agent QL and the state‐of‐the‐art equilibrium‐based MAQL algorithms. Then the cooperative control problem employing PGs mainly focusing upon the consensus problem is briefly discussed. The chapter also proposes a novel consensus QL (CoQL). Subsequently, a consensus‐based multirobot cooperative planning algorithm is proposed. Then two experiments are presented. The first experiment is designed to study the relative performance of the CoQL over the reference algorithms. Another experiment is framed to study the relative performance of the consensus‐based planning algorithm over the reference algorithms, considering multi‐robot stick carrying problem as a benchmark in terms of state‐transitions required to complete the task.