Notice of Violation of IEEE Publication Principles: Multiobjective Reinforcement Learning: A Comprehensive Overview

Notice of Violation of IEEE Publication Principles

"Multiobjective Reinforcement Learning: A Comprehensive Overview Authors"
by Chunming Liu, Xin Xu, and Dewen Hu
Submitted to IEEE Transactions on Cybernetics in May 2012

After careful and considered review of the content and authorship of this paper by a duly constituted expert committee, this submission has been found to not be in compliance with IEEE's Publication Principles.

This submission contains portions of original text from the papers cited below. While the authors reference the original papers, they did not sufficiently delineate the reused text from their own work and subsequently the submission was rejected from publication.

"Empirical Evaluation Methods for Multiobjective Reinforcement Learning Algorithms",
by Peter Vamplew, Richard Dazeley, Adam Berry, Rustam Issabekov, and Evan Dekker
in Machine Learning (2011) 84, pp. 51-80.

"On the Limitations of Scalarisation for Multi-Objective Reinforcement Learning of Pareto Fronts"
by Peter Vamplew, John Yearwood, Richard Dazeley, and Adam Berry
in Lecture Notes in Computer Science, vol. 5360, 2008, pp. 372-378.

"Multiple-goal Reinforcement Learning with Modular Sarsa(0)."
by Nathan Sprague and Dana Ballard
in Proceedings of the International Joint Conference on Artificial Intelligence, vol. 18.
Lawrence Erlbaum Associates Ltd., Technical Report 798, 2004.

A revised version of this submission was published in the IEEE Transactions on Systems, Man, and Cybernetics: Systems, Vol. 45, Issue 3, pp. 385-398

http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6918520
Reinforcement learning is a powerful mechanism for enabling agents to learn in an unknown environment, and most reinforcement learning algorithms aim to maximize some numerical value, which represents only one long-term objective. However, multiple long-term objectives are exhibited in many real-world decision and control problems; therefore, recently, there has been growing interest in solving multiobjective reinforcement learning (MORL) problems with multiple conflicting objectives. The aim of this paper is to present a comprehensive overview of MORL. The basic architecture, research topics, and naive solutions of MORL are introduced at first. Then, several representative MORL approaches and some important directions of recent research are reviewed. The relationships between MORL and other related research are also discussed, which include multiobjective optimization, hierarchical reinforcement learning, and multi-agent reinforcement learning. Finally, research challenges and open problems of MORL techniques are highlighted.