Referential Task-Oriented Emergent Semantic Communications for Mobile Augmented Reality

As mobile augmented reality (MAR) becomes integral to interacting with digital twins and the Metaverse, it demands stringent requirements in latency, computational efficiency, and energy conservation. Traditional methods leveraging edge, cloud computing, and 5G networks encounter considerable challenges, notably communication latency with visual data, which compromises the quality of experience (QoE). This paper presents an innovative semantic communication framework tailored for enhancing MAR applications. By adapting the Lewis signaling game, we train two agents to develop an efficient communication protocol, allowing the exchange of complex visual information via compact, meaningful messages. Unlike existing semantic communication models that emphasize data reconstruction, our approach prioritizes task completion, employing a discrete protocol optimized for referential tasks. To align with real-world scenarios, channel uncertainty is integrated into our training process. Experimental validations reveal that our method surpasses traditional semantic communications in referential task performance significantly, showcasing its potential for MAR applications while acknowledging the challenges of implementation. This work not only proposes a solution to a pressing problem in MAR but also opens avenues for future research in task-oriented communication.