Joint Sensing and Communication for mmWave VR in Metaverse: A Meta-Learning Approach

In this article, we propose a joint sensing and communication framework for virtual reality (VR) applications in Metaverse. Although millimeter-wave (mmWave) communication can achieve multi-Gbps wireless transmission data rate, a slight movement of the VR headset can result in a significant drop in transmission rate. This significantly deteriorates user’s experience in Metaverse applications. By characterizing the relationship between mmWave beam gain and beam width, we find that adaptively turning off part of antennas can improve the overall transmission performance for mobile Metaverse. To this end, we formulate a problem with the objective of adaptively configuring receiver’s phase shift, and adjusting the beam width to cope with the variation in VR user’s viewpoints in Metaverse services. By revealing the correlation between power consumption and signal-to-noise ratio of VR headset, the proposed dual method based on meta reinforcement learning enables reliable and energy-efficient mmWave communication for VR. Based on the sensing information collected from VR users, the beamforming strategy is continuously updated by reshaping the reward of learning process, which minimizes the power consumption while meeting transmission requirements of Metaverse applications. Extensive experimental results demonstrate that the adaptability of the proposed framework outperforms the existing benchmarks in various VR scenarios, which ensures the applicability of mmWave communication to Metaverse applications.