A method is presented for partitioning a scene into regions corresponding to surfaces with distinct velocities. Both motion and contrast information are incorporated into the segmentation process. Velocity estimates for each point in a scene are obtained using a local nonmatching technique not dependent on any prior boundary determination. The actual segmentation is accomplished using a region merging procedure which combines regions based on similarities in both brightness and motion. The method is effective in determining object boundaries not easily found using analysis applied only to a single image frame.