Extended Generative Adversarial Imitation Learning for Autonomous Agents in Minecraft Game

3D sandbox games like Minecraft allow users to perform innumerable actions in diverse and complex environments. The design of autonomous agents in various environments is crucial for real-world application and utilization. Generative adversarial imitation learning (GAIL) is designed to replicate human demonstrations in more complex and subtle environments and solve various tasks by autonomous agents, but it has limitations in processing complex sequence input and sufficiently suppressing incorrect policies, making it insufficient for generating complex actions for complex inputs in Minecraft. In this paper, we propose an extended GAIL that effectively addresses Minecraft tasks, considering the complexity of the inputs and outputs. The proposed method, using a global encoder shared between the agent and discriminator, ensures robustness for image sequences and improves the positive reward function to include negative values, enabling the agent to construct optimal trajectories among a variety of actions. Experiments on Minecraft game scenarios confirm the superiority of the proposed method over typical GAILs, achieving human-like scores in Navigate and TreeChop tasks. Additionally, we reveal that it is performed in a typical benchmark environment to show that it can operate in game domains. Ablation studies demonstrate that (1) the global encoder can extract meaningful representations from raw image sequence inputs, (2) the reward function significantly impacts the performance in composing actions available to the autonomous agent, and (3) the proposed method, through its adjustment of input and output in GAIL, can be applied across various GAIL-based models.