Certainty and expertness-based credit assignment for cooperative Q-Learning agents with an AND-type task

In multiagent reinforcement learning, inter-agent credit assignment is a fundamental problem, since a single scalar reinforcement signal is the only reliable feedback that teams of learning agents receive. This problem is more critical in groups of independent learners with a joint task. In this research, it is assumed that a critic agent receives the environment feedback and assigns a proper credit to each agent using some measures. Three of such measures for a team of cooperative agents with a parallel and AND-type task are introduced. These measures somehow compare the agents' knowledge. One of these criteria, called normal expertness, is a non-relative measure while two other ones (certainty and relative normal expertness) are relative measure. It is experimentally shown that relative measures work better as they contain more information for the critic agent.