Multi-Modal Stream Integrity Transmission Strategy for Multi-User Wireless Metaverse

The metaverse services are promising to embrace multi-sensory experiences of human beings, which mainly include audio-visual and tactile senses. From the perspective of wireless transmission, tactile transmission requires ultra-reliable low-latency communications, while audio-visual transmission requires enhanced mobile broadband communications. Besides, the audio-visual segment can be divided into several correlated data packets, any loss of packets would result in failed decoding at users, thus degrading users’ immersive experiences. In multi-user wireless metaverse systems, the heterogeneous transmission characteristics of multi-modal streams and integrity requirements of audio-visual stream transmission pose a great challenge to the limited wireless resource scheduling. To this end, we design a multi-user resource schedule scheme for multi-modal stream transmission by jointly considering the integrity of audio-visual stream transmission and the puncturing-based tactile stream transmission. We model the multi-modal perception utility function based on the multi-attribute utility theory and wireless transmission performance of multi-modal streams. Then, we formulate the average multi-modal perception utility maximization problem, and we adopt the Lyapunov theory to decompose the original maximization problem. Furthermore, we integrate the matching-based two-timescale spectrum resource allocation algorithm and alternating direction method of multipliers-based power allocation algorithm to obtain the optimal spectrum and power allocation strategies. Simulation results show that, compared with the resource allocation scheme without considering the transmission integrity, the average multi-modal perception utility of the proposed scheme is maximumly improved by 25%.