Non-rigid objects detection and segmentation in video sequence using 3D mean shift analysis

A new technique for robust detection and segmentation of non-rigid objects in video sequence is proposed in this paper. In our approach, spatio-temporal mean shift analysis (MSA) is employed to convert raw video/object data to their corresponding 3D/2D region feature spaces (RFS) respectively. The distance metric of RFS can be defined based on its spatio-temporal continuous property. Within the MSA derived RFS, any selected or given objects can be detected and segmented automatically in successive frames by local motion estimation. Experiments on various sequences show our method is robust to clutter, partial occlusion, object's out-of-plane rotation and significant relative movement among targets, scene and camera.