Using Markovian decision problems to analyze animal performance in random and variable ratio schedules of reinforcement

Markovian decision problems are a kind of optimization problems in which an agent must learn how to optimize the amount of reward it can collect during its interaction with its environment. We use them to analyze the task faced by an animal in random and variable schedules of reinforcement. Predictions of the model derived from this analysis are compared to three sets of data obtained in men, rats and pigeons and are contrasted with the ones of its main challenger in psychology, Herrnstein's equation. This reveals the existence of two response strategies in ratio schedules, one which corresponds to our model, the other which is closer to Herrnstein's equation