APCAFlow: All-Pairs Cost Volume Aggregation for Optical Flow Estimation

Optical flow estimation is a fundamental task in computer vision. The all-pairs correlation volume has enabled state-of-the-art performance in many optical flow estimation methods. However, all-pairs correlations provide only local matching clues, and lack global context, which could lead to mismatches in textureless and occluded regions. In this paper, we propose a novel all-pairs correlation volume aggregation (APCA) method which includes two key innovations. The first is a cost volume splitting and reassembling approach which partitions the full cost volume into smaller blocks and re-arranges those blocks to allow the use of 2D and 3D convolutions for cost volume aggregation. The second is hierarchical aggregation which performs 2D convolutions within blocks for local matching aggregation and 3D convolutions across blocks for global matching aggregation. We further design a novel optical flow estimation network APCAFlow based on APCA. APCAFlow achieves comparable performance to the most advanced approach, FlowFormer, but with significantly lower complexity. Specifically, APCAFlow reduces the model parameters, inference time, and memory consumption by 24.1%, 35.5%, and 21.6%, respectively, compared to FlowFormer. Furthermore, APCA can be easily integrated into several existing all-pairs cost volume-based methods for performance improvement. Code is available at 
https://github.com/MiaoJieF/APCAFlow
.