Eyehearyou: Probing Location Identification Via Occluded Smartphone Cameras and Ultrasound

This paper explores how to localize a device equipped solely with camera sensors by leveraging the unintended response of occluded camera hardware to transmitted ultrasound-specifically, determining with high probability which ultrasound transmission pattern was injected during image capture, based on distinctive ultrasound signals unwittingly detected by the image sensor. Prior device location identification methods require the use of dedicated hardware or protocols, e.g., microphone arrays or GPS. We envision a new potential mechanism by leveraging ubiquitous camera hardware on mobile and IoT devices to receive acoustic signals produced by ultrasonic localization beacons, even when the camera is occluded. We discover that ultrasonic signals affect gyroscopes integral to modern camera sensors' stabilization hardware and induce distinct destabilization signals in the dark images captured by occluded cameras. This work provides theoretical analysis, simulation modeling, and experimental evidence of how this optical acoustic side channel creates different noise patterns in camera images when the camera is subjected to different ultrasound stimuli. Our evaluation with 119 videos captured by a smartphone camera over multiple days shows success in detecting whether the smartphone is near an ultrasonic transmitter that can be associated with different locations.