Cooperative caching for scalable video coding using value-decomposed dimensional networks

Scalable video coding (SVC) has been widely used in video-on-demand (VOD) service, to efficiently satisfy users' different video quality requirements and dynamically adjust video stream to time-variant wireless channels. Under the 5G network structure, we consider a cooperative caching scheme inside each cluster with SVC to economically utilize the limited caching storage. A novel multi-agent deep reinforcement learning (MADRL) framework is proposed to jointly optimize the video access delay and users' satisfaction, where an aggregation node is introduced helping individual agents to achieve global observations and overall system rewards. Moreover, to cope with the large action space caused by the large number of videos and users, a dimension decomposition method is embedded into the neural network in each agent, which greatly reduce the computational complexity and memory cost of the reinforcement learning. Experimental results show that: 1) the proposed value-decomposed dimensional network (VDDN) algorithm achieves an obvious performance gain versus the traditional MADRL; 2) the proposed VDDN algorithm can handle an extremely large action space and quickly converge with a low computational complexity.