Action Space-Independent Exploration Methods in Multi-Agent Deep Reinforcement Learning for Wireless Power Allocation

Multi-agent deep reinforcement learning has pre-dominantly been applied to tackle the challenges of power allocation within complex and dynamic wireless networks. These intelligent agents rely on exploration as a fundamental tool to gather crucial information about their environment, reducing uncertainty and facilitating more informed power allocation decisions. Importantly, as a wireless channel is inherently dynamic, exploration is an indispensable mechanism for agents to continually adapt by acquiring fresh environment insights. Traditionally, most research has leaned towards employing semi-uniform distributed exploration for discrete action spaces and noise perturbation exploration for continuous action spaces. In this paper, we set out to question the validity of this default choice. We modify both exploration methods to work in both action spaces and with different exploration speeds. We demonstrate that semi-uniform distributed exploration yields comparable results to noise perturbation exploration in a continuous action space while significantly simplifying the exploration factor's fine-tuning. In contrast, noise perturbation exploration excels primarily in continuous action space.