Using 3D structure and anisotropic diffusion for object segmentation

We describe an advanced technique that can be employed to carry out the segmentation task. The goal is the development of a system capable of solving the segmentation problem in most situations encountered in video sequences taken from real world scenes. For this aim the presented segmentation scheme comprises different processing steps implemented as independent modules. The core of the system is a module for multiscale image simplification by anisotropic diffusion. The remaining modules are concerned with the subsequent image segmentation of the resulting smoothed images. The mathematical model supporting the implemented algorithms is based on the numerical solution of a system of nonlinear partial differential equations introduced by Perona and Malik (1987). The idea at the heart of this approach is to smooth the image in direction parallel to the object boundaries, inhibiting diffusion across the edges. The goal of this processing step is to enhance edges keeping their correct position, reducing noise and smoothing regions with small intensity variations. The techniques for object segmentation presented are based on image simplification by nonlinear diffusion and subsequent extraction of object masks taking into account disparity or motion fields as additional information. Two different strategies are considered in order to extract masks of physical objects in the scene: depth-driven nonlinear diffusion and edge extraction and enhancement in scale-space followed by edge matching in two different views of the same scene.