MPOT: Manifold Preserving Optimal Transport for Visual Recognition Under Severe Distribution Shift

Optimal transport (OT) is a rising research area to overcome distribution shifts in real-world data, which has been widely applied in visual signal processing tasks due to its appealing mathematical properties. However, previous works 1) consider the transport cost in Euclidean space, which conflicts with the well-known manifold prior on intrinsic data structure; 2) only consider first-order relation between inputs, which is infeasible for severe shift with heterogeneous spaces. These limitations usually disrupt the manifold structure and degrade the generalization performance on the test data. To deal with these issues, we propose the manifold preserving OT (MPOT) on Gromov-Wasserstein (GW), which introduces 1) the graph-based cost formulation for high-order relation characterization; 2) relation modeling for unshared knowledge between heterogeneous spaces. Mathematically, by encoding the high-order edge information as binary pattern, the GW-based regularization is developed to capture the intrinsic structure for label discriminability. Numerical algorithm with theoretical guarantee is provided, which ensures that MPOT can be efficiently solved by block coordinate descent. Extensive experiments validate MPOT for cross-domain visual classification with changing label spaces.