A Black Box Approach to Inferring, Characterizing, and Breaking Native Device Tracking Autonomy

Autonomous capabilities have emerged as a ubiquitous technology in cars, robots, drones and many other useful devices that are destined to transform our society. Many of the more advanced autonomous devices use computer vision to track and recognize objects, while others may use LIDAR or SONAR or other forms of radar. Since many of these devices are positioned to be disruptive technologies in our society, the fidelity of their abilities is paramount. In this paper, we introduce our black box approach, which can accurately infer, characterize, and break native device autonomy. We demonstrate this approach by using the tightly guarded autonomy code in the immensely popular computer vision driven autonomous DJI drones (i.e., ActiveTrack Mode) as the target native autonomy. We illustrate that this method allows us to quickly infer detailed information about the computer vision tracking algorithm, characterize the autonomous capability, and identify targets that defy these algorithms. We posit that this approach can be extended to other computer vision driven autonomous systems and is a necessary step in developing nondestructive privacy preserving mechanisms that foil the presumed tracking process.