A Case Study of Security and Privacy Threats from Augmented Reality (AR)

In this paper, we present a case study of potential security and privacy threats from virtual reality (VR) and augmented reality (AR) devices, which have been gaining popularity. We introduce a computer vision-based attack using an AR system based on the Samsung Gear VR and ZED stereo camera against authentication approaches on touch-enabled devices. A ZED stereo camera is attached to a Gear VR headset and works as the video feed for the Gear VR. While pretending to be playing, an attacker can capture videos of a victim inputting a password on touch screen. We use depth and distance information provided by the stereo camera to reconstruct the attack scene and recover the victim's password. We use numerical passwords as an example and perform experiments to demonstrate the performance of our attack. The attack's success rate is 90% when the distance from the victim is 1.5 meters and a reasonably good success rate is achieved within 2.5 meters. The goal of the paper is to raise awareness of potential security and privacy breaches from seemingly innocuous VR and AR technologies.