An Efficient Computing of Correlated Equilibrium for Cooperative Q‐Learning‐Based Multi‐Robot Planning

This chapter introduces a novel approach to adapt composite rewards of all the agents in one Q‐table in joint state‐action space during learning, and uses these rewards to compute correlated equilibrium in the planning phase. The ΩQ‐learning algorithms proposed in this chapter have two attractive features, which are not available in the traditional correlated Q‐learning (CQL). First, during the learning phase, an agent needs to adapt only one Q‐table in joint state—action space unlike adapting
m
joint Q‐tables for
m
agents in CQL. Second, the evaluation of the computationally expensive correlated equilibrium is avoided, following a tricky approach of computing it partially during the learning and the rest during the planning phases. The chapter provides an overview of the single agent Q‐learning and equilibrium‐based multi‐agent Q‐learning (MAQL) algorithms. Proposed cooperative MAQL and corresponding planning algorithms are then provided. The chapter offers complexity analysis, and presents simulation and experimental results.