Abstract:
With the development of deep reinforcement learning, much progress in various perfect and imperfect information games has been achieved. Among these games, DouDizhu, a po...Show MoreMetadata
Abstract:
With the development of deep reinforcement learning, much progress in various perfect and imperfect information games has been achieved. Among these games, DouDizhu, a popular card game in China, poses great challenges because of the imperfect information, large state and action space as well as the cooperation issue. In this article, we put forward an AI system for this game, which adopts opponent modeling and coach-guided training to help agents make better decisions when playing cards. Besides, we take the bidding phase of DouDizhu into consideration, which is usually ignored by existing works, and train a bidding network using Monte Carlo simulation. As a result, we achieve a full version of our AI system that is applicable to real-world competitions. We conduct extensive experiments to evaluate the effectiveness of the three techniques adopted in our method and demonstrate the superior performance of our AI over the state-of-the-art DouDizhu AI, i.e., DouZero. We upload our AI systems, one is bidding-free and the other is equipped with a bidding network, to Botzone platform and they both rank the first among over 400 and 250 AI programs on the two corresponding leaderboards, respectively.
Published in: IEEE Transactions on Games ( Volume: 16, Issue: 3, September 2024)
Funding Agency:
CAS Key Laboratory of Technology in Geo-spatial Information Processing and Application System, Department of Electronic Engineering and Information Science, University of Science and Technology of China, Hefei, China
Youpeng Zhao (Graduate Student Member, IEEE) received the B.E. degree in electronic engineering and information science in 2021 from the University of Science and Technology of China (USTC), Hefei, China, where he is currently working toward the M.E. degree in information and communication engineering with the Department of Electronic Engineering and Information Science.
His research interests include reinforcement learnin...Show More
Youpeng Zhao (Graduate Student Member, IEEE) received the B.E. degree in electronic engineering and information science in 2021 from the University of Science and Technology of China (USTC), Hefei, China, where he is currently working toward the M.E. degree in information and communication engineering with the Department of Electronic Engineering and Information Science.
His research interests include reinforcement learnin...View more
CAS Key Laboratory of Technology in Geo-spatial Information Processing and Application System, Department of Electronic Engineering and Information Science, University of Science and Technology of China, Hefei, China
Jian Zhao received the Ph.D. degree in electronic engineering and information science from the University of Science and Technology of China, Hefei, China, in 2023.
His research interests include reinforcement learning and multi-agent system.
Jian Zhao received the Ph.D. degree in electronic engineering and information science from the University of Science and Technology of China, Hefei, China, in 2023.
His research interests include reinforcement learning and multi-agent system.View more
CAS Key Laboratory of Technology in Geo-spatial Information Processing and Application System, Department of Electronic Engineering and Information Science, University of Science and Technology of China, Hefei, China
Xunhan Hu received the M.E. degree in electronic engineering and information engineering from the University of Science and Technology of China (USTC), Hefei, China, in 2023.
Her research interests include reinforcement learning and multi-agent system.
Xunhan Hu received the M.E. degree in electronic engineering and information engineering from the University of Science and Technology of China (USTC), Hefei, China, in 2023.
Her research interests include reinforcement learning and multi-agent system.View more
CAS Key Laboratory of Technology in Geo-spatial Information Processing and Application System, Department of Electronic Engineering and Information Science, University of Science and Technology of China, Hefei, China
Institute of Artificial Intelligence, Hefei Comprehensive National Science Center, Hefei, China
Wengang Zhou (Senior Member, IEEE) received the B.E. degree in electronic information engineering from Wuhan University, Wuhan, China, in 2006, and the Ph.D. degree in electronic engineering and information science from University of Science and Technology of China (USTC), Hefei, China, in 2011.
From 2011 to 2013, he worked as a Postdoc Researcher with the Computer Science Department, University of Texas at San Antonio, Sa...Show More
Wengang Zhou (Senior Member, IEEE) received the B.E. degree in electronic information engineering from Wuhan University, Wuhan, China, in 2006, and the Ph.D. degree in electronic engineering and information science from University of Science and Technology of China (USTC), Hefei, China, in 2011.
From 2011 to 2013, he worked as a Postdoc Researcher with the Computer Science Department, University of Texas at San Antonio, Sa...View more
CAS Key Laboratory of Technology in Geo-spatial Information Processing and Application System, Department of Electronic Engineering and Information Science, University of Science and Technology of China, Hefei, China
Institute of Artificial Intelligence, Hefei Comprehensive National Science Center, Hefei, China
Houqiang Li (Fellow, IEEE) received the B.S., M.Eng., and Ph.D. degrees in electronic engineering from the University of Science and Technology of China, Hefei, China, in 1992, 1997, and 2000, respectively.
He is currently a Professor with the Department of Electronic Engineering and Information Science. He has authored and coauthored over 200 papers in journals and conferences. His research interests include reinforcement...Show More
Houqiang Li (Fellow, IEEE) received the B.S., M.Eng., and Ph.D. degrees in electronic engineering from the University of Science and Technology of China, Hefei, China, in 1992, 1997, and 2000, respectively.
He is currently a Professor with the Department of Electronic Engineering and Information Science. He has authored and coauthored over 200 papers in journals and conferences. His research interests include reinforcement...View more
Contents CAS Key Laboratory of Technology in Geo-spatial Information Processing and Application System, Department of Electronic Engineering and Information Science, University of Science and Technology of China, Hefei, China
Youpeng Zhao (Graduate Student Member, IEEE) received the B.E. degree in electronic engineering and information science in 2021 from the University of Science and Technology of China (USTC), Hefei, China, where he is currently working toward the M.E. degree in information and communication engineering with the Department of Electronic Engineering and Information Science.
His research interests include reinforcement learning and multi-agent systems.
Youpeng Zhao (Graduate Student Member, IEEE) received the B.E. degree in electronic engineering and information science in 2021 from the University of Science and Technology of China (USTC), Hefei, China, where he is currently working toward the M.E. degree in information and communication engineering with the Department of Electronic Engineering and Information Science.
His research interests include reinforcement learning and multi-agent systems.View more
CAS Key Laboratory of Technology in Geo-spatial Information Processing and Application System, Department of Electronic Engineering and Information Science, University of Science and Technology of China, Hefei, China
Jian Zhao received the Ph.D. degree in electronic engineering and information science from the University of Science and Technology of China, Hefei, China, in 2023.
His research interests include reinforcement learning and multi-agent system.
Jian Zhao received the Ph.D. degree in electronic engineering and information science from the University of Science and Technology of China, Hefei, China, in 2023.
His research interests include reinforcement learning and multi-agent system.View more
CAS Key Laboratory of Technology in Geo-spatial Information Processing and Application System, Department of Electronic Engineering and Information Science, University of Science and Technology of China, Hefei, China
Xunhan Hu received the M.E. degree in electronic engineering and information engineering from the University of Science and Technology of China (USTC), Hefei, China, in 2023.
Her research interests include reinforcement learning and multi-agent system.
Xunhan Hu received the M.E. degree in electronic engineering and information engineering from the University of Science and Technology of China (USTC), Hefei, China, in 2023.
Her research interests include reinforcement learning and multi-agent system.View more
CAS Key Laboratory of Technology in Geo-spatial Information Processing and Application System, Department of Electronic Engineering and Information Science, University of Science and Technology of China, Hefei, China
Institute of Artificial Intelligence, Hefei Comprehensive National Science Center, Hefei, China
Wengang Zhou (Senior Member, IEEE) received the B.E. degree in electronic information engineering from Wuhan University, Wuhan, China, in 2006, and the Ph.D. degree in electronic engineering and information science from University of Science and Technology of China (USTC), Hefei, China, in 2011.
From 2011 to 2013, he worked as a Postdoc Researcher with the Computer Science Department, University of Texas at San Antonio, San Antonio, TX, USA. He is currently a Professor with the EEIS Department, USTC his research interests, which include multimedia information retrieval, computer vision, and computer game.
Dr. Zhou was the recipient of the Best Paper Award for ICIMCS 2012. He was the Publication Chair of IEEE ICME 2021.
Wengang Zhou (Senior Member, IEEE) received the B.E. degree in electronic information engineering from Wuhan University, Wuhan, China, in 2006, and the Ph.D. degree in electronic engineering and information science from University of Science and Technology of China (USTC), Hefei, China, in 2011.
From 2011 to 2013, he worked as a Postdoc Researcher with the Computer Science Department, University of Texas at San Antonio, San Antonio, TX, USA. He is currently a Professor with the EEIS Department, USTC his research interests, which include multimedia information retrieval, computer vision, and computer game.
Dr. Zhou was the recipient of the Best Paper Award for ICIMCS 2012. He was the Publication Chair of IEEE ICME 2021.View more
CAS Key Laboratory of Technology in Geo-spatial Information Processing and Application System, Department of Electronic Engineering and Information Science, University of Science and Technology of China, Hefei, China
Institute of Artificial Intelligence, Hefei Comprehensive National Science Center, Hefei, China
Houqiang Li (Fellow, IEEE) received the B.S., M.Eng., and Ph.D. degrees in electronic engineering from the University of Science and Technology of China, Hefei, China, in 1992, 1997, and 2000, respectively.
He is currently a Professor with the Department of Electronic Engineering and Information Science. He has authored and coauthored over 200 papers in journals and conferences. His research interests include reinforcement learning, multimedia search, image/video analysis, and video coding and communication.
Dr. Li is the winner of National Science Funds for Distinguished Young Scientists, the Distinguished Professor of Changjiang Scholars Program of China, and the Leading Scientist of Ten Thousand Talent Program of China. He is an Associate Editor (AE) for IEEE transactions on multimedia and was the AE of IEEE transactions on circuits and systems for video technology. He was the General Co-Chair of ICME 2021 and the TPC Co-Chair of VCIP 2010. He was the recipient of the Best Paper Award for ACM MUM in 2011, Best Paper Award for VCIP 2012, National Natural Science Award of China (second class) in 2015, and National Technological Invention Award of China (second class) in 2019.
Houqiang Li (Fellow, IEEE) received the B.S., M.Eng., and Ph.D. degrees in electronic engineering from the University of Science and Technology of China, Hefei, China, in 1992, 1997, and 2000, respectively.
He is currently a Professor with the Department of Electronic Engineering and Information Science. He has authored and coauthored over 200 papers in journals and conferences. His research interests include reinforcement learning, multimedia search, image/video analysis, and video coding and communication.
Dr. Li is the winner of National Science Funds for Distinguished Young Scientists, the Distinguished Professor of Changjiang Scholars Program of China, and the Leading Scientist of Ten Thousand Talent Program of China. He is an Associate Editor (AE) for IEEE transactions on multimedia and was the AE of IEEE transactions on circuits and systems for video technology. He was the General Co-Chair of ICME 2021 and the TPC Co-Chair of VCIP 2010. He was the recipient of the Best Paper Award for ACM MUM in 2011, Best Paper Award for VCIP 2012, National Natural Science Award of China (second class) in 2015, and National Technological Invention Award of China (second class) in 2019.View more
