An Imbalanced Mean-Field Game Theoretical Large-Scale Multiagent Optimization With Constraints

A novel optimization algorithm has been developed for distributed large-scale multiagent systems (LS-MASs), specifically focusing on achieving a terminal density constraint. While the recent advancement in mean field game (MFG) offers a feasible distributed solution to address the “Curse of Dimensionality” problem, it compromises the optimality of large-scale homogeneous agents and lacks the capability to achieve arbitrary fixed terminal probability density function (PDF) constraint, especially when deviating from the normal distribution. To tackle this issue, a novel approach called the imbalanced mean-field game (Imb-MFG) theory has been designed alongside an adaptive PDF decomposition method and distributed reinforcement learning (RL) that can effectively obtain optimal solutions in LS-MAS even with fixed terminal density constraints in a distributed manner. In particular, a method based on the induction theory has been developed for estimating the parameter of the final PDF constraint, enabling the decomposition of MFG-PDF into multiple imbalanced normal distributions. Subsequently, the Imb-MFG theory is developed by integrating the decomposed multigroup MFG agents with a K-means clustering algorithm with constraint. The developed Imb-MFG approach decomposes a single PDF into multiple imbalanced normal distributions, which are combined to achieve arbitrary terminal PDF constraints. To achieve the solution of the Imb-MFG theory, a multiactor–critic-mass (M-ACM) algorithm is developed. This algorithm is developed to concurrently learn the solution for coupled Fokker–Planck–Kolmogorov (FPK) and Hamilton–Jacobi–Bellman equations. The algorithm’s convergence is ensured through the Lyapunov analysis. The effectiveness of this algorithm is validated through a simulation study.