Mobile Edge Adversarial Detection for Digital Twinning to the Metaverse: A Deep Reinforcement Learning Approach

Digital Twinning of physical world scenes onto the Metaverse is necessary for augmented reality (AR)-assisted driving. In AR-assisted driving, physical environment scenes are first captured by AR vehicles and are uploaded to the Metaverse for the construction of the Metaverse Map. However, the development of AR-assisted driving applications invites adversaries. These attackers may place adversarial patches on physical objects, seeking to contort the Metaverse Map. As real-time, accurate detection of adversarial patches is compute-intensive, these physical world scenes have to be offloaded to the Metaverse Map Base Station (MMBS) for computation. Therefore, we considered a scenario where AR vehicles capture physical world scenes and upload these scenes in real-time to the MMBSs. We formulated an optimization problem where the MMSP’s objective is to maximize adversarial patch detection mean Average Precision (mAP), while minimizing the computed AR scene uplink transmission latency and minimizing the worst-case (largest) AR vehicle’s uplink transmission battery charge consumption, through optimizing the AR vehicle-MMBS allocation, AR vehicle uplink scene resolution selection, and AR vehicle uplink power output selection. We proposed a Heterogeneous Action (HA) algorithm to tackle the proposed problem. Extensive experiments show our HA models outperforms baseline models when compared against key metrics.