MAKE Decision in Hybrid Action Space

Although hybrid action space problems are prevalent in real-life scenarios, research on hybrid action space reinforcement learning algorithms has historically been less popular in the field of reinforcement learning. In this paper, we propose an algorithmic framework for actor-critic style reinforcement learning in hybrid action spaces. We further develop a novel algorithm within this framework, utilizing the Q-learning policy optimization approach. To demonstrate the effectiveness of our approach, we extensively evaluate our algorithm through a series of experiments conducted in the RoboCup2D Half Field Football (HFO) environment. Our approach has achieved 2 percentage point improvement in task success rate compared to previous studies, with an average reduction of approximately 13% in the number of time steps required for each successful trajectory.