Hormonal systems for prisoners dilemma agents

A large number of studies have evolved agents to play the iterated prisoners dilemma. This study models a novel type of interaction, called the Shopkeeper model of interaction, in which a state conditioned agent interacts with a series of other agents without resetting its internal state. This is intended to simulate the situation in which a shopkeeper interacts with a series of customers. This study is the second to use the shopkeeper model and uses a more complex agent representation for the shopkeepers. In addition to a finite state machine for play, the shopkeepers are equipped with an artificial hormonal system that retains information about recent play. This models the situation in which an agent's current behaviour is affected by its previous encounters. We train hormonal shopkeeper prisoners dilemma agents against a variety of distributions of possible customers. The shopkeepers are found to specialize their behavior to their customers, but often fail to discover maximally exploitative behaviors against more complex customer types. This study introduces a technique called agent-case embeddings to demonstrate that shopkeepers adapt to their customers and that hormonal and non-hormonal agents are different. Agent-case embeddings are representation-independent maps from agents behaviors into Euclidean space that permit analysis across different representations. Unlike an earlier tool, fingerprinting, they can be applied easily to very complex agent representations. We find that the hormonal shopkeepers exhibit a substantially different distribution of evolved strategies than the non-hormonal ones. Additionally, the evolved hormonal systems exhibit a variety of hormone level patterns demonstrating that multiple types of hormonal systems evolve.