Regional Video Object Segmentation by Efficient Motion-Aware Mask Propagation

The use of optical flow to aid feature matching has been employed in recent self-supervised video object segmentation (VOS) methods and has shown promising results. However, computing pixel-wise optical flow is costly, and the optical flow can also be further utilized for efficient regional segmentation. To address these challenges, we propose an efficient motion-aware mask propagation approach, dubbed EMMP, for self-supervised VOS. EMMP introduces an efficient patch optical flow to compute the motion offsets of image patches for dynamic matching ROI generation. Fine-grained pixel-wise feature matching is performed based on the dynamic matching ROIs for mask propagation. To reduce redundant segmentation while avoiding unnecessary computations, we re-use the patch optical flow to estimate reliable foreground ROIs in the next frame and perform regional segmentation. Evaluation on benchmark VOS datasets shows that EMMP achieves competitive performance with significant wall-clock speed-ups compared to existing self-supervised training methods, e.g., EMMP slightly outperforms MAMP and runs about 2× faster on segmentation. In addition, EMMP performs on par with many supervised training methods.