Deformable Cross Attention for Learning Optical Flow

Optical flow is the process of estimating motion in scenes. Each object in the scene has a homogeneous motion, i.e., moves in the same direction with the same velocity. Therefore, connecting the parts of an image globally provides an essential cue for learning accurate motion. Convolution-based methods estimate the motion features from the local regions, which miss this important cue. Recently, some methods used Transformer to model global dependencies to improve optical flow. However, Transformer suffers from excessive attention computations and still brings irrelevant parts into the region of interest. Therefore, we propose a deformable cross-attention for optical flow estimation, which provides two important advantages: connecting the parts of the image globally while deforming the attention to the objects’ shapes in the image and reducing the memory consumption. Our proposed method achieved competitive performance on Sintel and KITTI 2015 datasets in terms of accuracy and efficiency.