Abstract:
In multiagent reinforcement learning (MARL), it is crucial for each agent to model the relation with its neighbors. Existing approaches usually resort to concatenate the ...Show MoreMetadata
Abstract:
In multiagent reinforcement learning (MARL), it is crucial for each agent to model the relation with its neighbors. Existing approaches usually resort to concatenate the features of multiple neighbors, fixing the size and the identity of the inputs. But these settings are inflexible and unscalable. In this article, we propose an attentive relational encoder (ARE), which is a novel scalable feedforward neural module, to attentionally aggregate an arbitrary-sized neighboring feature set for state representation in the decentralized MARL. The ARE actively selects the relevant information from the neighboring agents and is permutation invariant, computationally efficient, and flexible to interactive multiagent systems. Our method consistently outperforms the latest competing decentralized MARL methods in several multiagent tasks. In particular, it shows strong cooperative performance in challenging StarCraft micromanagement tasks and achieves over a 96% winning rate against the most difficult noncheating built-in artificial intelligence bots.
Published in: IEEE Transactions on Cybernetics ( Volume: 52, Issue: 1, January 2022)
Funding Agency:
Key Laboratory of Machine Perception (Ministry of Education) and Department of Machine Intelligence, School of Electronics Engineering and Computer Science, Peking University, Beijing, China
Xiangyu Liu received the Bachelor of Engineering degree in intelligent science and technology from Nankai University, Tianjin, China, in 2015. He is currently pursuing the Ph.D. degree in computer science with the Key Laboratory of Machine Perception (Ministry of Education), Department of Machine Intelligence, EECS, Peking University, Beijing, China.
His research interests include multiagent reinforcement learning, swarm i...Show More
Xiangyu Liu received the Bachelor of Engineering degree in intelligent science and technology from Nankai University, Tianjin, China, in 2015. He is currently pursuing the Ph.D. degree in computer science with the Key Laboratory of Machine Perception (Ministry of Education), Department of Machine Intelligence, EECS, Peking University, Beijing, China.
His research interests include multiagent reinforcement learning, swarm i...View more
Key Laboratory of Machine Perception (Ministry of Education) and Department of Machine Intelligence, School of Electronics Engineering and Computer Science, Peking University, Beijing, China
Ying Tan (Senior Member, IEEE) received the B.Eng., M.S., and Ph.D. degrees from Southeast University, Nanjing, China, in 1985, 1988, and 1997, respectively.
He is a Full Professor and a Ph.D. Advisor with the School of EECS, and the Director of Computational Intelligence Laboratory, Peking University, Beijing, China. He is the inventor of Fireworks Algorithm. He worked as a Professor with the Faculty of Design, Kyushu Uni...Show More
Ying Tan (Senior Member, IEEE) received the B.Eng., M.S., and Ph.D. degrees from Southeast University, Nanjing, China, in 1985, 1988, and 1997, respectively.
He is a Full Professor and a Ph.D. Advisor with the School of EECS, and the Director of Computational Intelligence Laboratory, Peking University, Beijing, China. He is the inventor of Fireworks Algorithm. He worked as a Professor with the Faculty of Design, Kyushu Uni...View more
Contents Key Laboratory of Machine Perception (Ministry of Education) and Department of Machine Intelligence, School of Electronics Engineering and Computer Science, Peking University, Beijing, China
Xiangyu Liu received the Bachelor of Engineering degree in intelligent science and technology from Nankai University, Tianjin, China, in 2015. He is currently pursuing the Ph.D. degree in computer science with the Key Laboratory of Machine Perception (Ministry of Education), Department of Machine Intelligence, EECS, Peking University, Beijing, China.
His research interests include multiagent reinforcement learning, swarm intelligence, and swarm robotics.
Xiangyu Liu received the Bachelor of Engineering degree in intelligent science and technology from Nankai University, Tianjin, China, in 2015. He is currently pursuing the Ph.D. degree in computer science with the Key Laboratory of Machine Perception (Ministry of Education), Department of Machine Intelligence, EECS, Peking University, Beijing, China.
His research interests include multiagent reinforcement learning, swarm intelligence, and swarm robotics.View more
Key Laboratory of Machine Perception (Ministry of Education) and Department of Machine Intelligence, School of Electronics Engineering and Computer Science, Peking University, Beijing, China
Ying Tan (Senior Member, IEEE) received the B.Eng., M.S., and Ph.D. degrees from Southeast University, Nanjing, China, in 1985, 1988, and 1997, respectively.
He is a Full Professor and a Ph.D. Advisor with the School of EECS, and the Director of Computational Intelligence Laboratory, Peking University, Beijing, China. He is the inventor of Fireworks Algorithm. He worked as a Professor with the Faculty of Design, Kyushu University, Fukuoka, Japan, in 2018, a Senior Research Fellow with Columbia University, New York, NY, USA, in 2017, and a Research Fellow with the Chinese University of Hong Kong, Hong Kong, in 1999 and from 2004 to 2005. His research interests include computational intelligence, swarm intelligence, deep neural networks, machine learning, data mining, and intelligent information processing for information security and financial prediction. He has published more than 350 papers in refereed journals and conferences in these areas, and authored/coauthored 12 books, including Fireworks Algorithm (Springer in 2015), and GPU-Based Parallel Implementation of Swarm Intelligence Algorithm (Morgan Kaufmann and Elsevier in 2016), and received five invention patents.
Prof. Tan won the Second-Class Natural Science Award of China in 2009 and the Second-Class Natural Science Award of Ministry of Education of China in 2019 and many best paper awards. He serves as the Editor-in-Chief for the IASEI Transactions on Swarm Intelligence and the International Journal of Computational Intelligence and Pattern Recognition, and an Associate Editor for the IEEE Transactions on Cybernetics, the IEEE Transactions on Neural Networks and Learning System, Neural Networks, and the International Journal of Swarm Intelligence Research. He also served as an Editor of Springer’s Lecture Notes on Computer Science for 40+ volumes, and a guest editors of several referred journals, including the IEEE/ACM Transactions on Computational Biology and Bioinformatics, Information Science, Neurocomputing, Natural Computing, and Swarm and Evolutionary Optimization. He has been the Founding General Chair of the ICSI international conference series since 2010 and the DMBD conference series since 2016.
Ying Tan (Senior Member, IEEE) received the B.Eng., M.S., and Ph.D. degrees from Southeast University, Nanjing, China, in 1985, 1988, and 1997, respectively.
He is a Full Professor and a Ph.D. Advisor with the School of EECS, and the Director of Computational Intelligence Laboratory, Peking University, Beijing, China. He is the inventor of Fireworks Algorithm. He worked as a Professor with the Faculty of Design, Kyushu University, Fukuoka, Japan, in 2018, a Senior Research Fellow with Columbia University, New York, NY, USA, in 2017, and a Research Fellow with the Chinese University of Hong Kong, Hong Kong, in 1999 and from 2004 to 2005. His research interests include computational intelligence, swarm intelligence, deep neural networks, machine learning, data mining, and intelligent information processing for information security and financial prediction. He has published more than 350 papers in refereed journals and conferences in these areas, and authored/coauthored 12 books, including Fireworks Algorithm (Springer in 2015), and GPU-Based Parallel Implementation of Swarm Intelligence Algorithm (Morgan Kaufmann and Elsevier in 2016), and received five invention patents.
Prof. Tan won the Second-Class Natural Science Award of China in 2009 and the Second-Class Natural Science Award of Ministry of Education of China in 2019 and many best paper awards. He serves as the Editor-in-Chief for the IASEI Transactions on Swarm Intelligence and the International Journal of Computational Intelligence and Pattern Recognition, and an Associate Editor for the IEEE Transactions on Cybernetics, the IEEE Transactions on Neural Networks and Learning System, Neural Networks, and the International Journal of Swarm Intelligence Research. He also served as an Editor of Springer’s Lecture Notes on Computer Science for 40+ volumes, and a guest editors of several referred journals, including the IEEE/ACM Transactions on Computational Biology and Bioinformatics, Information Science, Neurocomputing, Natural Computing, and Swarm and Evolutionary Optimization. He has been the Founding General Chair of the ICSI international conference series since 2010 and the DMBD conference series since 2016.View more
