SKFlow: Optical Flow Estimation Using Selective Kernel Networks

Leveraging on the recent developments in convolutional neural networks (CNNs), optical flow estimation from adjacent frames has been cast as a learning problem, with performance exceeding traditional approaches. The existing networks always use standard convolutional layers for extracting multi-level features with the fixed kernel size at each level. For enlarging the receptive field, some works introduce dilated convolution operation, which can capture more contextual information and can avoid the loss of motion details. However, these networks lack the ability to adaptively adjust its receptive field size and cannot aggregate multi-scale information with a selective mechanism. To address this problem, in this paper, we introduce selective kernel network into optical flow estimation, which can adaptively select different scale features and adjust their receptive field according to the global information. Specifically, we conduct the selective kernel mechanism on each level of pyramid, which can adaptively select multi-scale feature at each pyramidal level. The extensive analyses are conducted on MPI-Sintel and KITTI datasets to verify the effectiveness of the proposed approach. The experimental results show that our model achieves comparable results with the previous state-of-the-art networks while keeping a small model size.