Hierarchical hidden Markov modeling for team-play in multiple agents

When we consider imitation learning of team-play in multi-agent systems, we need to define a suitable building block and its interface to construct complex joint behaviors. I focus on agent intentions as the building block that abstracts local situations of the agent, and propose a hierarchical hidden Markov model (HMM). The key of the proposed model is introduction of gate probabilities that restrict transitions among agents' intentions according to others' intentions. Using these probabilities, the framework can control transitions flexibly among basic behaviors in a cooperative behavior. Experiments shows that the model can acquire suitable timing to change an agent's intention cooperatively with other agents.