Sequentially updated Probability Collectives

Multi-agent coordination problems can be cast as distributed optimization tasks. Probability collectives (PCs) are techniques that deal with such problems in discrete and continuous spaces. In this paper we are going to propose a new variation of PCs, sequentially updated probability collectives. Our objective is to show how standard techniques from the statistics literature, sequential Monte Carlo methods and kernel regression, can be used as building blocks within PCs instead of the ad hoc approaches taken previously to produce samples and estimate values in continuous action spaces. We test our algorithm in three different simulation scenarios with continuous action spaces. Two classical distributed optimization functions, the three and six dimensional Hartman functions and a vehicle target assignment type game. The results for the Hartman functions were close to the global optimum, and the agents managed to coordinate to the optimal solution of the target assignment game.