AR Marker Hiding with Real-Time Texture Deformation

Augmented Reality (AR) marker hiding is a technique to visually remove AR markers from a real-time video stream. The conventional approach transforms a background image with a homography matrix calculated on the basis of a camera pose and overlays the transformed image on the AR marker region in the real-time frame, assuming that the AR marker is on a planar surface. However, this approach can cause discontinuities in textures around the boundary between the marker and its surrounding area when the planar surface assumption is not satisfied. This paper proposes a method for AR marker hiding without discontinuities around texture boundaries under unknown and nonplanar background geometry. The proposed method estimates dense motion in the marker's background based on feature point correspondences around the marker, together with a smooth motion assumption, and deforms the background image according to the dense motion. Experimental results demonstrate the effectiveness of the proposed method in various environments with different background geometries and textures.