Synthesizing Six Years of AR/VR Research: A Systematic Review of Machine and Deep Learning Applications

Augmented Reality (AR), Virtual Reality (VR), and Mixed Reality (MR), when combined with machine learning (ML) and deep learning (DL), represent both challenging and promising research areas. However, there is currently a lack of comprehensive surveys reviewing their contributions. In this review paper, we present a thorough analysis of the most recent research on AR/VR/MR applications with ML and DL models in the IEEE VR and ISMAR conferences from 2018 to 2023. Our literature review process, which involved multiple filtering steps, resulted in 154 relevant publications focusing on ML/DL. The paper covers a broad spectrum of topics, including object recognition, tracking, segmentation, depth estimation, 3D reconstruction, and interactive systems. We highlight the significant contributions of ML/DL and their potential impact on the AR/VR/MR fields and provide a curated list of publicly available datasets ¹ from AR/VR/MR environments to support further research. This review offers a valuable resource for researchers and practitioners interested in the latest advancements and future directions in ML/DL applications within AR/VR/MR technologies. Additionally, we discuss emerging research trends and challenges, providing insights into the opportunities for future work in these fields.