Image segmentation combining region depth and object features

Object recognition systems need image segmentation processes that relate image regions to world objects. These methods present often three problems: the generation of a large number of small regions, undersegmentation (different objects are associated to the same image region) and oversegmentation (a scene object is segmented in various regions). In order to overcome these problems, we propose an image segmentation method that combines depth information and object surface properties obtained from a pair of stereo images. The system work under the standard assumption that 3D objects have planar faces and regular shapes. First a region growing segmentation process is applied to both images generating two labeled images. Then, depth information of the region frontiers is obtained by matching the labeled segments from left and right image rows. The stereo matching problem is solved by finding a path through a 2D search plane whose axes are the left and right segmented lines. Original image regions are then merged based on their size, surface information and frontiers depth information. In this way, image regions are associated to surfaces that are contiguous in the 3D space and they present a common property (such as gray level, color or texture).