A novel approach to depth ordering in monocular image sequences

Object oriented representation of image sequences requires accurate motion segmentation and depth ordering techniques. Unfortunately, the lack of precise motion estimates at the object boundaries makes these two tasks very difficult. We present a detailed analysis of the behaviour of dense motion estimation techniques at object boundaries which reveals the systematic nature of the motion estimation error; the motion of the occluding surface is observed in a small neighbourhood on the occluded side. We then show how the joint use of still image segmentation and robust regression can eliminate this error. Furthermore we present a novel technique which uses the position of the error as a depth cue. The validity of this technique, which requires only sub-pixel motion and which is capable of distinguishing between different types of intensity discontinuities, such as object boundaries, surface marks and illumination discontinuities, is then demonstrated on several synthetic and real image sequences.