Sensitivity-based inverse reinforcement learning

Inverse reinforcement learning (IRL) is a process to obtain a potential reward function according to expert's behavior. Then the optimal control policy is generated though some optimization theory, such as reinforcement learning, so that we can implement the imitation for expert's behavior. In this paper, we consider the inverse reinforcement learning principle from t he point of performance sensitivity analysis. After that, we propose a novel inverse reinforcement learning analytical framework by analyzing the performance difference formula between expert's policy and any other policies. This analytical framework extends the standard inverse reinforcement learning to the case that the reward function is related with both states and actions. At the same time, this framework provides a unified approach for IRL with the discount reward and the average reward in Markov decision process. Finally, the validity of corresponding results is verified under a grid world problem.