Handling occlusion in augmented reality systems: a semi-automatic method

We present a semi-automatic approach to solve occlusion in AR systems. Once the occluding objects have been segmented by hand in selected views called key-frames, the occluding boundary is computed automatically in the intermediate views. To do that, the 3D reconstruction of the occluding boundary is achieved from the outlined silhouettes. This allows us to recover a good prediction of the 2D occluding boundary which is refined using region-based tracking and active contour models. As a result, we get an accurate estimation of the occluding objects. Various results are presented demonstrating occlusion resolution on real video sequences. Results and videos are available at the URL: http://www.loria.fr/-lepetit/Occlusions.