MoiréTracker: Continuous Camera-to-Screen 6-DoF Pose Tracking Based on Moiré Pattern

In the realm of AR applications and particularly camera-to-screen interactions, camera tracking stands as a crucial technology. However, the ever-increasing demand for tracking accuracy makes it essential to explore a six-degrees of freedom (6-DoF) tracking technology with ultra-high precision to facilitate micro-motion sensing. In this paper, we propose a novel sensing method MoiréTracker to achieve camera’s 6-DoF pose tracking with ultra-high precision. MoiréTracker outputs camera’s continuous 3-DoF trajectory and 3-DoF posture changes according to the captured moiré patterns, which can be produced by the superposition of camera’s Color Filter Array (CFA) and the projection of screen raster on the CFA plane. Thanks to moiré pattern’s high sensitivity to 6-DoF motions, we characterize the relationship between moiré features and camera’s micro pose changes, so as to realize the continuous 6-DoF pose tracking for camera with ultra-high precision. Moreover, our proposal involves a thumbnail-based method aimed at expanding the working range of MoiréTracker, enabling the pervasive camera-to-screen interactions. We implement a prototype system and evaluate its performance in real-world environments. Extensive experiment results show that MoiréTracker achieves the average trajectory error of 1.20 cm and the posture error of 1.07°.