Power Efficient Networking Support for Digital Twins with Age of Information Targets

This paper studies network resource allocation for multiple IoT devices as physical systems (PSs) tasked with maintaining their digital twins (DTs) at a shared edge server (ES) through a shared communication channel. The problem is formulated as a constrained Markov decision process with an objective of reducing the average transmission power of the PSs while keeping the age of information (Aol) at the DTs below predetermined targets. A hybrid decision making frame is proposed, where multiple agent reinforcement learning is used to make decisions for the transmission power of the PSs in a distributed way, and a centralized and deterministic algorithm is proposed to allocate the computation resources of the ES among the DTs. Simulation results show that, compared with the multiagent duelling double deep Q-Network, the proposed multi-agent deep deterministic policy gradient for power allocation together with the urgency-baed computation resource allocation solution achieves much lower the average power consumption of the PSs while maintaining low AoI violation rate at the DTs.