Learning Stigmergic Communication for Self-organising Coordination

Self-organisation in multi-agent systems (MAS) requires agents to coordinate their actions according to the specific goal to be pursued by the system as a whole. When agents can fully observe their peers, coordination can be implicit, that is, relying solely on appropriate reactions to others’ actions and their effects. When observability is limited, instead, explicit communication is needed. In this case, it is often taken as a built-in capability of agents, and the communication policy is defined at design time. But learning to communicate may enable the MAS to better adapt to novel deployments, run-time changes, and/or changing goals. In this paper, we study learning stigmergic communication in a MAS whose self-organising goal changes between aggregating agents in clusters and its opposite— scattering them around. Through experiments in NetLogo we show four results: (i) learning to communicate is possible, and can also be more effective than built-in communication; (ii) agents learn to self-organise so that the "correct" (i.e. leading to goal achievement) global sequencing of actions arises (i.e. few agents deposit pheromones while most follow pheromone trails, in the case of aggregation goal); and (iii) learning communication policies instead of having them built-in enables self-(re)organisation towards a new goal without heavy intervention in the learning process itself (i.e. by simply changing the reward scheme).