An Arms Race in Theory-of-Mind: Deception Drives the Emergence of Higher-level Theory-of-Mind in Agent Societies

It has been argued that using Theory-of-Mind (ToM) is a fundamental cognitive ability that underpins effective deceptive behaviour. However, we still do not have a clear understanding of the effect ToM has on deception under evolutionary pressure in self-organising multi-agent systems (MAS). To better understand it, in this paper we introduce the Evolutionary Deception Game (EDG) to model deception and investigation of deception in societies of agents with different levels of ToM. We show that the impact of higher levels of ToM is different in deceptive agents compared to agents that investigate deception. We also show the creation of a dominance cycle, i.e., an arms race in ToM where investigators need to be one level higher to dominate deceivers. Additionally, we provide new insight into the effect that important societal factors in MAS such as social learning and population size have on the dominance of strategies in the EDG. We conclude that ToM is both the solution to agent deception, as well as its defence.