Abstract:
Through well-designed optimization paradigm and deep neural networks as feature extractor, deep reinforcement learning (DRL) algorithms learn optimal policy on discrete a...Show MoreMetadata
Abstract:
Through well-designed optimization paradigm and deep neural networks as feature extractor, deep reinforcement learning (DRL) algorithms learn optimal policy on discrete and continuous action space. However, such capability is restricted by the low sampling efficiency. By inspecting the importance of feature extraction in DRL, we find that state feature learning is one of the key obstacles for sampling efficiently. To this end, we propose a new state representation learning scheme with adjacent state consistency loss (ASC loss). The loss is based on the hypothesis that the distance between adjacent states is smaller than that of far apart ones since scenes in videos generally evolve smoothly. We exploit ASC loss as an assistant of RL loss in the training phase to boost the state feature learning, and make evaluation on existing DRL algorithms as well as behavioral cloning algorithm. Experiments on Atari games and MuJoCo continuous control tasks demonstrate the effectiveness of our scheme.
Published in: IEEE MultiMedia ( Volume: 28, Issue: 3, 01 July-Sept. 2021)
Funding Agency:
CAS Key Laboratory of GIPAS, Electronic Engineering and Information Science Department, University of Science and Technology of China, Hefei, China
Tianyu Zhao is currently working toward the master’s degree in electronics and communication engineering with the Department of Electronic Engineering and Information Science, University of Science and Technology of China, Hefei, China. His research interests include deep learning and reinforcement learning. Zhao received the B.E. degree in bioinformatics from the Harbin Institute of Technology, Harbin, China, in 2019. Co...Show More
Tianyu Zhao is currently working toward the master’s degree in electronics and communication engineering with the Department of Electronic Engineering and Information Science, University of Science and Technology of China, Hefei, China. His research interests include deep learning and reinforcement learning. Zhao received the B.E. degree in bioinformatics from the Harbin Institute of Technology, Harbin, China, in 2019. Co...View more
CAS Key Laboratory of GIPAS, Electronic Engineering and Information Science Department, University of Science and Technology of China, Hefei, China
Jian Zhao is currently working toward the Ph.D. degree in information and communication engineering with the Department of Electronic Engineering and Information Science, University of Science and Technology of China (USTC), Hefei, China. His research interests include computer vision and reinforcement learning. Zhao received the B.E. degree in electronic engineering and information science from the USTC, in 2018. Contact...Show More
Jian Zhao is currently working toward the Ph.D. degree in information and communication engineering with the Department of Electronic Engineering and Information Science, University of Science and Technology of China (USTC), Hefei, China. His research interests include computer vision and reinforcement learning. Zhao received the B.E. degree in electronic engineering and information science from the USTC, in 2018. Contact...View more
CAS Key Laboratory of GIPAS, Electronic Engineering and Information Science Department, University of Science and Technology of China, Hefei, China
Wengang Zhou is currently a Professor with the Electronic Engineering and Information Science Department, University of Science and Technology of China (USTC), Hefei, China. His research interests include multimedia information retrieval and computer vision. Zhou received the B.E. degree in electronic information engineering from Wuhan University, Wuhan, China, in 2006, and the Ph.D. degree in electronic engineering and i...Show More
Wengang Zhou is currently a Professor with the Electronic Engineering and Information Science Department, University of Science and Technology of China (USTC), Hefei, China. His research interests include multimedia information retrieval and computer vision. Zhou received the B.E. degree in electronic information engineering from Wuhan University, Wuhan, China, in 2006, and the Ph.D. degree in electronic engineering and i...View more
CAS Key Laboratory of GIPAS, Electronic Engineering and Information Science Department, University of Science and Technology of China, Hefei, China
Yun Zhou is currently a Postdoctoral Researcher with the Electronic Engineering and Information Science Department, University of Science and Technology of China, Hefei, China. Her research interests include computer vision and reinforcement learning. Zhou received the B.S degree in electronic and information engineering from Anhui Normal University, Wuhu, China, in 2010, and the Ph.D. degree in communication and informat...Show More
Yun Zhou is currently a Postdoctoral Researcher with the Electronic Engineering and Information Science Department, University of Science and Technology of China, Hefei, China. Her research interests include computer vision and reinforcement learning. Zhou received the B.S degree in electronic and information engineering from Anhui Normal University, Wuhu, China, in 2010, and the Ph.D. degree in communication and informat...View more
CAS Key Laboratory of GIPAS, Electronic Engineering and Information Science Department, University of Science and Technology of China, Hefei, China
Houqiang Li (Fellow, IEEE) is currently a Professor with the Department of Electronic Engineering and Information Science, University of Science and Technology of China, Hefei, China. His research interests include image/video analysis, computer vision, reinforcement learning, etc. He has authored and coauthored more than 200 papers in journals and conferences. Li received the B.S., M.Eng., and Ph.D. degrees in electronic...Show More
Houqiang Li (Fellow, IEEE) is currently a Professor with the Department of Electronic Engineering and Information Science, University of Science and Technology of China, Hefei, China. His research interests include image/video analysis, computer vision, reinforcement learning, etc. He has authored and coauthored more than 200 papers in journals and conferences. Li received the B.S., M.Eng., and Ph.D. degrees in electronic...View more
Contents CAS Key Laboratory of GIPAS, Electronic Engineering and Information Science Department, University of Science and Technology of China, Hefei, China
Tianyu Zhao is currently working toward the master’s degree in electronics and communication engineering with the Department of Electronic Engineering and Information Science, University of Science and Technology of China, Hefei, China. His research interests include deep learning and reinforcement learning. Zhao received the B.E. degree in bioinformatics from the Harbin Institute of Technology, Harbin, China, in 2019. Contact him at zhty@mail.ustc.edu.cn. He is the corresponding author of this article.
Tianyu Zhao is currently working toward the master’s degree in electronics and communication engineering with the Department of Electronic Engineering and Information Science, University of Science and Technology of China, Hefei, China. His research interests include deep learning and reinforcement learning. Zhao received the B.E. degree in bioinformatics from the Harbin Institute of Technology, Harbin, China, in 2019. Contact him at zhty@mail.ustc.edu.cn. He is the corresponding author of this article.View more
CAS Key Laboratory of GIPAS, Electronic Engineering and Information Science Department, University of Science and Technology of China, Hefei, China
Jian Zhao is currently working toward the Ph.D. degree in information and communication engineering with the Department of Electronic Engineering and Information Science, University of Science and Technology of China (USTC), Hefei, China. His research interests include computer vision and reinforcement learning. Zhao received the B.E. degree in electronic engineering and information science from the USTC, in 2018. Contact him at zj140@mail.ustc.edu.cn.
Jian Zhao is currently working toward the Ph.D. degree in information and communication engineering with the Department of Electronic Engineering and Information Science, University of Science and Technology of China (USTC), Hefei, China. His research interests include computer vision and reinforcement learning. Zhao received the B.E. degree in electronic engineering and information science from the USTC, in 2018. Contact him at zj140@mail.ustc.edu.cn.View more
CAS Key Laboratory of GIPAS, Electronic Engineering and Information Science Department, University of Science and Technology of China, Hefei, China
Wengang Zhou is currently a Professor with the Electronic Engineering and Information Science Department, University of Science and Technology of China (USTC), Hefei, China. His research interests include multimedia information retrieval and computer vision. Zhou received the B.E. degree in electronic information engineering from Wuhan University, Wuhan, China, in 2006, and the Ph.D. degree in electronic engineering and information science from the USTC, in 2011. From September 2011 to 2013, he was a Postdoctoral Researcher with the Computer Science Department, University of Texas at San Antonio. Contact him at zhwg@ustc.edu.cn.
Wengang Zhou is currently a Professor with the Electronic Engineering and Information Science Department, University of Science and Technology of China (USTC), Hefei, China. His research interests include multimedia information retrieval and computer vision. Zhou received the B.E. degree in electronic information engineering from Wuhan University, Wuhan, China, in 2006, and the Ph.D. degree in electronic engineering and information science from the USTC, in 2011. From September 2011 to 2013, he was a Postdoctoral Researcher with the Computer Science Department, University of Texas at San Antonio. Contact him at zhwg@ustc.edu.cn.View more
CAS Key Laboratory of GIPAS, Electronic Engineering and Information Science Department, University of Science and Technology of China, Hefei, China
Yun Zhou is currently a Postdoctoral Researcher with the Electronic Engineering and Information Science Department, University of Science and Technology of China, Hefei, China. Her research interests include computer vision and reinforcement learning. Zhou received the B.S degree in electronic and information engineering from Anhui Normal University, Wuhu, China, in 2010, and the Ph.D. degree in communication and information system from Hefei University of Technology, Hefei, China, in 2018. Contact her at zhouyun@ustc.edu.cn.
Yun Zhou is currently a Postdoctoral Researcher with the Electronic Engineering and Information Science Department, University of Science and Technology of China, Hefei, China. Her research interests include computer vision and reinforcement learning. Zhou received the B.S degree in electronic and information engineering from Anhui Normal University, Wuhu, China, in 2010, and the Ph.D. degree in communication and information system from Hefei University of Technology, Hefei, China, in 2018. Contact her at zhouyun@ustc.edu.cn.View more
CAS Key Laboratory of GIPAS, Electronic Engineering and Information Science Department, University of Science and Technology of China, Hefei, China
Houqiang Li (Fellow, IEEE) is currently a Professor with the Department of Electronic Engineering and Information Science, University of Science and Technology of China, Hefei, China. His research interests include image/video analysis, computer vision, reinforcement learning, etc. He has authored and coauthored more than 200 papers in journals and conferences. Li received the B.S., M.Eng., and Ph.D. degrees in electronic engineering from the University of Science and Technology of China, in 1992, 1997, and 2000, respectively. He is the winner of National Science Funds (NSFC) for Distinguished Young Scientists, the Distinguished Professor of Changjiang Scholars Program of China, and the Leading Scientist of Ten Thousand Talent Program of China. He was an Associate Editor for the IEEE Transactions on Circuits and Systems for Video Technology from 2010 to 2013. He was the TPC Co-Chair of VCIP 2010, and he will serve as the General Co-Chair of ICME 2021. He was the recipient of the National Technological Invention Award of China (second class) in 2019 and the National Natural Science Award of China (second class) in 2015. He was also the recipient of the Best Paper Award for VCIP 2012, the Best Paper Award for ICIMCS 2012, and the Best Paper Award for ACM MUM in 2011. Contact him at lihq@ustc.edu.cn.
Houqiang Li (Fellow, IEEE) is currently a Professor with the Department of Electronic Engineering and Information Science, University of Science and Technology of China, Hefei, China. His research interests include image/video analysis, computer vision, reinforcement learning, etc. He has authored and coauthored more than 200 papers in journals and conferences. Li received the B.S., M.Eng., and Ph.D. degrees in electronic engineering from the University of Science and Technology of China, in 1992, 1997, and 2000, respectively. He is the winner of National Science Funds (NSFC) for Distinguished Young Scientists, the Distinguished Professor of Changjiang Scholars Program of China, and the Leading Scientist of Ten Thousand Talent Program of China. He was an Associate Editor for the IEEE Transactions on Circuits and Systems for Video Technology from 2010 to 2013. He was the TPC Co-Chair of VCIP 2010, and he will serve as the General Co-Chair of ICME 2021. He was the recipient of the National Technological Invention Award of China (second class) in 2019 and the National Natural Science Award of China (second class) in 2015. He was also the recipient of the Best Paper Award for VCIP 2012, the Best Paper Award for ICIMCS 2012, and the Best Paper Award for ACM MUM in 2011. Contact him at lihq@ustc.edu.cn.View more
