Multi-agent Reinforcement Learning based Distributed Channel Access for Industrial Edge-Cloud Web 3.0

In the emerging Web 3.0 applications for mass customized and personalized manufacturing, smart mobile resources need to interact with each other and other resources to achieve efficient collaborative manufacturing. Existing wireless communication solutions cannot leverage multi-antenna technology and the movement direction of smart mobile resources to meet the high requirements for communication rate and reliability in high-performance manufacturing processes. Therefore, this paper proposes a task-aware distributed channel access scheme for multi-antenna smart mobile resources in a factory. First, this paper introduces an edge-cloud collaboration framework for smart factories to support autonomous wireless access point selection for mobile resources. Second, a user-centric active wireless channel access scheme is proposed and a channel resource allocation optimization problem is formulated for mobile resources to leverage multiple antennas and movement direction to address the unstable connection problem. Third, a centralized-training-and-distributed-execution multi-agent reinforcement learning (MARL) model with a specially designed neural network architecture is built for smart mobile resources, effectively using important input information of the next interaction objects for mobile resources. Simulation results show that the proposed MARL scheme outperforms common schemes of 3GPP LTE, traditional reinforcement learning schemes, and random selection schemes in improving communication rate and stability.