Scene segmentation from 3D motion

We propose an EM approach combined with the modified separation matrix scheme to perform 3D motion segmentation of the image sequence which contains multiple moving objects. We observe that, given the detected features and their 2D optical flow, in most cases the objects or their flow are separated very well from each other in space. The separation matrix method modified by using normalized cuts achieves expected grouping results for these cases. However, when the objects are overlapped spatially but undergoing independent motions, such as Ullman's co-axial transparent cylinder demonstration, there will be no proper affinity to perform segmentation by that way. Pure underlying 3D motion becomes the only cite to segment the scene. We exploit the EM algorithm to deal with such difficult cases. The scheme is tested on vast number of synthetic image sequences. Results with real image sequence are also given.